Exploring Explicit Coarse-Grained Structure in Artificial Neural Networks

We propose to employ a hierarchical coarse-grained structure in artificial neural networks explicitly improve the interpretability without degrading performance. The idea has been applied two situations. One is network called TaylorNet, which aims approximate general mapping from input data output result terms of Taylor series directly, resorting any magic nonlinear activations. other new setup for distillation, can perform multi-level abstraction dataset and generate that possesses relevant features original be used as references classification. In both cases, plays an important role simplifying improving efficiency. validity demonstrated on MNIST CIFAR-10 datasets. Further improvement some open questions related are also discussed.